Emotional Subtitles: A System and Potential Applications for Deaf and Hearing Impaired People

Many Deaf and Hearing Impaired people use subtitles to gain access to audio content on television and film presentations. Although subtitles tell the viewer what is being said they fail to communicate how it is being said. This “emotional gap” experienced by viewer’s highlights a significant drawback to current subtitling especially when used for learning by the Deaf. In this paper we introduce a system that demonstrates the presentation of subtitles that depict the emotions behind the words used on screen. The system also provides viewers with the ability to personalize and adapt their interaction with subtitles, so as to assist them in their learning. Using this system we hope to conduct a series of surveys looking at how people receive and use subtitles. In conducting this research we aim to gain a comprehensive understanding of the issues associated with emotional subtitling and to provide guidance for future producers of subtitled materials.